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ABSTRACT 

The use of of the Automatic Interaction Detector 
(program AIDS of the OSIRIS statistical package) to study a 
university program is discussed. The performance of students who took 
general physics lecture and laboratory concurrently is compared to 
the performance of those who took them separately. Five years of data 
are analyzed, covering 1,997 students. The educational outcomes 
investigation uses the AID3 program to examine a variety of 
independent variables, including Scholast ic Aptitude Test (SAT) 
scores, physics lecture grade, student major, student level, high 
school rank, and student sex. The AID3 analysis indicates that the 
importance of the concurrent laboratory depends strongly on when in 
the analysis it is considered. The demographic measures that make a 
difference in explaining grades in physics class include: overall 
grade average, SAT math score, high school rank, and the year in 
which the student took the course. Finally, some nonadditive 
interactions occur in the data. It is demonstrated that the research 
method allows researchers to consider a large number of variables as 
possible explainers of course performance without making the 
restrictive assumptions required by other techniques. (SW) 
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"AiL)"-iNii acadkmh: imukikam kvai.uatidn 

■i;hr "Aulom.iLif liilt.'i-.iiM.ioii :)i't,iH;lc)i-" A:'. An;i I y i r> Tool 



liilrodui.'t.ion I'lt.';. eooLi V(j:_i 



AoiKl..'iniL^ pi-oj.'.ram:.; 'irc; mofu IIk.iii (;vcr cHi.led into qucsLion. InsL u.ul luiinl 
lu.';u.';u\.'l' i;-. i iicr-ear. i iiL'.i Y asked to provide iiiipar L 1 al analyses m Lhe;;e i.uiLLers, 
(•si.oo ialiy w»:oii a pfocrai.i is subjcel. to str-onyly lieLo coiiiM ictiiif, views. A 
.;aso in poll'., is t'.c'iieral physios iaboratocy . Many afgue tlwit this ro.-.ource- 
oonsuMier is lamecessar y , that students can get the same educational benclit 
via other means. IH can be in an awkward position when askcJ to "referee." 
Analytic methods which have served may now be inappropriate or even narmful . 



Pur po.'U' 

M.iny factors contribute to an edueatiunal outcome, pt«riiaps too many to be 
amenable' to traditional nietliods of statistical analysis. Traditionally one 
would use a stepwise proeeoure with reL,ression or discriminate analysis. 
Unfortunately such analyses assume that the interactions of the explanatory 
variables are additive and that important interactions are with the variables 
already included in the model. The "Automatic Interaction Detector," an iter- 
ative branehinti algorithm, (Institute for Social Research, iy73, PP- bb'i-'jbe) 
is used in -.his study as an alternative method which does not require us to 
make these assumptions. It also provides the researcher with insight as to 
Khici. of many possible independent variables arc best at explaining vai'iance 
in the dependent variable. 

Tne "Automatic Interaction Detector" (program "AID3" of the "USIHIS" sta- 
tistical package) is unique among statistical analysis tools. It is more akin 
to a chess-playing computer program. At a given point (node) in its analysis, 
it "looks ahead" to see uhich possible variable combinations lead to which 
possible statistical significance. It then chooses the path that maximizes 
significance. In doing so, it constructs a hierarchical "tree" structure. 
Nodes at levels nearest the "root" represent the most significance. Variables 
of the least effect are automatically filtered out to the farthest "branches" 
of the tree. 

The value of AID3 in an ejucational outcomes analysis is obvious. It can 
"chew on" the widest possible set of explanatory variables and "spit out" all 
but tne best. With A1D3 there is no need to guess at the best sot or to use 
only those variatles that are "obviously" appropriate for a good analysis. 
AID3 automaticallv provides the best set, which may then be used m more trad- 
itional statistical analyses. Others (Cohen, 1983) have used AID3 alone for 
analyses The present work used AID3 in combination with more traditional 
analysis tools, as e "front. end" to winnow the possible explanatpry variables 
to a best set of manageable size. 

U_tera_t ^ re Rev iew 

A recent article (Toothacker, 1983) ■ presents the results of several stu- 
dies related to the objectives of general jcience laboratory (Dubravcic,, 1979; 
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AlUirih', r, ok: mill ^valuation 



K,.u,.-l,-,l.- ir,,'- Kn,K,l„k. Ui',^; :;auii.lcM';; l)H;l;.:i.:.;uri . I'JY'J) i<"i .■o„r 1 u.h.;; >.!>;. I. 
,1,,'y .•••il'l Uir rl imiti.it.ioii of :i'...-,limat. atui .■.npliomofc pliyr, w:;-, Uil.urMl.o, m':, . 

•n„. lU,crat.uro not. v,o (H„-;',uh.M. how... v or . oUior sU.u ,10.-. tiavo Ooon nuMn,- 
rlor.ivo (lu.lLor.K.or. iy()<j; Cu,, ni horn , l'J'l6; I'owor . IV/o; :'.lu,lman ami 
IMY-J- W.il.-.on 196-^. Yet oUior f i tul i riK,:; have fifiiily .-.upix)!- loci tlio roio oi uio 
Lool'ro/ laboratory cotiiln nation ( llfailJ oy . Vjf.b. p. f.:.'; Cuiumi|',ham , I'i'KO . Ail 
coulr. bo viowod a:; fl awod . with too r.mall ,,.x po r ui.nnta I population;;., too i rvi or 
too liiany ck.p.ndont v ar .1 ah L . too littlo oonoern for "llawthorno" • , 

(Dubr.-ivoic. 19Yy; Kruglak. Kru,,.,lak. Vjbi; Saundorr. i Dickon:;,on. 19/))- 



Data oour*i:ti^ 



At Virrinia Tech. it is not neeaa.ary that a student take Ceneral 'hy:.ics 
lecture and^aborator y concurrently. As a result, sitnificant nnmoers do not. 
T.nis one can compare the lecture per lonnance of those students who took the 
j„;,.,rat.ory ooncurrontly and those who did not. from r-outme university .'.rado 
reports. This study used five years of such data, coverinc ;^.1»^> , , 

enrollments. 27 percent of which were for students who did not enroll n the 
concurrent laboratory. At a gross level, the data were us shown lu lable 1 



below . 



Table__l_ 

Concurrent Laboratory Work and Student Grades in Physics Lecture 



Academic Term 



Lab Students 
Num . Grade** 



Non-Lab students (Irade 
Num . Grade Dili . 



Fall 1979 1 


350 2.08 1 


1 

^23 1.61 1 0.11 


Fsl.l 1980 


3it7 2.17 1 


138-'~- 1-63 1 0.31 


Fall 1981 


2.59 


125 ^'-07 1 0.52 

1 


Fall 1982 


313 2.39 


95 2.09 1 0-30 


Fall 1983 


1 215 2.17 


106 2.10 1 0.07 
1 


Combined 


1 1.599 2.29 


] 587 1-89 ; 0.10 

1 

[ 1 . 



«» Average grade is calculated on the basis that an "A" is 1.00. 

At the gross level, it is apparent that students ^fin^ laboratory concur- 
rently have historically done better that "non-lao" ^ f^, ^,^'"", 
aouarent that things began changing in 1982. There is currently little dU- 
•?e?ence in Physi^cs lecture grade between the "lab" and "non-lab" students. 

With a large population and with the ability to use unobtrusiv e measures . 
condifi ns were'almost ideal for this type of ^P^-^,^^-^^-;^': ,"^3^,°'^^ 

investigation. With the power of AID3. a broad varie'.y ol independent varia- 



A3I)i iiK Propj'i'Hfi Kv Ml nation 



\\\r'.\ could Im' i nvc:;1. i K.'it,*'(i l.n .inr.wnr* (\uv.i\l\o\\ nV llie pfol)<il) L i, n 1' 1 iJ(Mujo oi' 

I al)()i\il(u-y un k rM(»*''» riiy'-^t^:*' .)()iir:u:. l)Mi{|u|.w\tph i and aiMd.cni ic rxplana- 

lovy var- i al) 1 r:-. viorc drawn i'\-u\n ttu^ uii i vc?r\'; i ly dala l)a:;r and added Lo t,li(' r,rad(: 
diil.a. Tin* .'-.liidcnU variabhir. W(M't' llu» I'ollowitiK: 



] . riiyn . I.(.'cL . (Ir-adc 

n . ::>AT Malh r>oort» 

y. OviM\aIl (li'adc Ave 

10. otudunU Major 



;\ Ccaiour r tail I.al> 

'). I>AT Verbal b^.'oi'e 

. :;iudcMa \,cvci 

1 1 Tranal'er lUudeta 



\ . Year •ana; 1 .1 i.'d 

b . Ilil'li r>c:}iO(j.l lianl 

9. Knlcjr iaui Level 

i;>. .Student Sex 



he'caiK^e ol" AIDS'.'^ non-additive natur-e, one may inelude variable:'. Ibat ..'oiild 
hardly be' tliuu^bl i\v Mv~iiVi\icA/ini^, . A:.; the above lli.;t r.buw::, one may Ibr'ow in 
everything, ineiudim^ the kitehen aiaik. The total population ol' student 
enrol leea'wa:; rednc^ed to l/jyy beenur.e r>ome :Uaidents did not receive "A" 
throuL',h "F" letter i^rades. 

{)i\r.v. tdie in;'>t ilait ionaJ data had been i'^-^^ I^hyiJiea eiaa> 1 I '/e.s . 

tliey W'.M'e taN.in:Uated inlo the tmmer'ie (jo(\cij recjuired by AiUj. 'Ihi:: la an 
extra :;tep. and it it; a minor nuisance. A simple SAS pi'Oi^,raiii converted the 
data from "institutional form to tiunieric as I'ollows: 



A 12. B-f 1 1 . B 10, etc 

79, 80. 81 . 83 

0 NA, 1 <^>00. Z yjOO 

1 NA. 2 C-2. 3 2-3. ^4 3- 
1 Fr . 2 3o , 3 . ^1 Sr 

Xf*er : 0 No . 1 Yes 



1. (ird: 
3. Yr: 
b. SATV: 
7. QvGr: 
9. FLvl : 
11 . 



2. Lab: 1 Lab. 0 No-Lab 

i\. ^ATM: 0 NA. 1 <500. 2 >1^00 

6. HSR: Class Hank Decile 

b. SLvl : 1 Fr . 2 So. 3 , M Sr 

10. Majr: 1 Ag . 2 Ar . 3 Bu . M Ed 

12. Sex : 0 Female. 1 Male 



Ln 



The abbreviations in the above conversion list correspond to the data items 
investigated (by number). The abbreviations will be used in this paper's oata 
tables and figures. 

Mc^Uiod 

Witli A1D3 we performed two separate analyses on the data. The I'irst ana- 
lysis was unconstrained, considering all of the variables to be of the same 
importance. Under this procedure, the algorithm selects the variabxe which 
when used to split the sample gives the largest reduction in the ^'between" sum 
of squares. This unconstrained procedure investigates the general question of 
what variables do the best job of explaining Physics lecture grades. 

The second procedure took advantage of AID3*s ability to constrain an 
analysis Since the immediate qu. otion involved the value of concurrent Phy- 
sics laboratory, the algorithm fc/ced the first division on the '• Lab/No-Lab" 
variable. Further splits were unconstrained. 

AID3 analysis yields a hierarchy of explanatory variables. The hierarchy 
may best be viewed as an "inverted tree.'* as shown in Figures 1 and 2. The 
tree diagram for the case where LAB is not forced as the first split is shown, 
in Figure 1 The keys to interpreting the diagram are the node numbers and 
the hierarchical levels. The lower the node number, the more significant the 
explanatory variable. The nearer a level is to the root, the more significant 
are that level's variables. Going down three levels (a good rule of thun- . 
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— . — ___ AiniiiR rroKi'-'iiii i.v.iiuiii. uMi 

...T-:!-.,, h.ysH.-. i,..t,nr.. Kra.!'.:.. H .1 uln M, Un n. v.,y , 
.•urr.nt, 1 .aK.,-„t,or y al all. Tn i snrt, ol r.:'.nM, , nol, ivm.al y wh , h- 

n.,1 picture. A. will be Ur. ..mhhuI pa:;. - w.U, U..' In,.!. „i.l... 

o„ "l.ab/No-l.ab" - r.: vnluabK'. a.-, v;o 1 1 a:; pa 1 ,.t.,-,l. 1 o . 

The value of U.e eunMr a u.ed analy,.. > :'. -bv i -.u.-, . l-.r,ure / 
u^u.-rera. laboratory .a.n M' > cant L y related to the rby.u.. 

,,,ned by tbe .tudents. worth at least half a le.ter f ^ * 

similar aeademic ..tature. In addition to concurrent laboratory t ; | ^ 
V ,ur, that the other major explanatory variable, are - a. u. the u ..t . .,e 

: ove-all grade ave.-a,e. T.AT Math r.oore. the year tl,at the r.tnU.nt too,- ,.e 

(M.ur.st; , and liiiih .".ehooL rank. 

The re^-ults of the two runs of AID3 i"'ply l-l"-^---" ^'O''^^ >^--' ' 'I''';-' 

r„.st' :t the Lnportance of concurrent 1 ^^--'-'-y f ^ o^' ^'^ 

n the analysis it is considered. The second conclusion ''^ ' , ; 

iemcraphic ,neasu, - s which ,nake a ^'^n-erer.ce in exp .u. n, d 
SK..3 cJass include overall grade aye"a,e .AT lath non-addxt.ve 
U.e year in which ^j- ^^^d-r ^ ,. ^ '^s n'o.n notu.,, the sy.- 

interactions occur in these data. laio uD.-er /ai. lu, 
inetry of importance of the variables. 

K.-om the last observation also cornes the conclusion that relaUvel y eon- 

v'-y large nu... oi ceii^ correlated inaepeiident varia- 

vacant. 

«t this point in the analysis pr-o.-css, MDJ has served its „L,r,,ose. Fol- 
l„„„ th AIDS analyses, the ..oasures ..*.ch see..ed to be the «os relevant .h 

!-zes It would have also created the potential for a extremely complex set 
of "s;Bnificant" differences that would have been impossible to interpret. 
Pre-processing with AID3 avoided those -.problems . 
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riu:.' Wll.h l>u;-,t, line ,l,,l,,i. Al, l.llf .•;.UI|r t.lllir H. 1 Mil 1 » . I ' '„ , 1 1 1 (. Ml.lt. Wc li;,- Ulr 

\„.::\. i„,,-..-,ihl iiiclrl; in . I ; : r i pi. i v i ■ f.'.-.r.irrl. I.u iiiI'T t.li<' iiioImI' I <',iii;;r ol' 
I I CI ,,,-,,n,.|'.-, i: (.iil..M,iiH';, . Thi;-. ...nrh in.iKr;; l.wu cniit.r I l.nt. loii;; in invr;;. 

l.i,.,,l.ioM (>r I.: Imi:; which iiiriniMM:- .:<.nt;.'' |wir.Miii,ii I'lr.'ii, it. f c pi ,r I,;; on 

I!;,! ,-,p,.<M I'ic: ,ii"';'t- '"■ ' • ''''>'■■■'"'•'■■ l.'i'' 1" '■xpl.iiMMiK, l.l"' 

p,.,Tiirm;iiu:r .... .■■.l.u-lciit..,-. in .1 l'hy;wc;; .'(nir . " A;; iiol..'.! in l\u.: rrvi.'W ni' Uu- 
1 il,.'iMt.urr. tin:; i ;■. .'uiTi.nl. I y .'in i.-,.';uc of coic-'ni wil.hiii l.hi.' I'hy:; ic". i-i-of.T,- 
i on . 
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,i..„Hn;;t.iMlion ol ,1 m.'l.hod whioh ;illow;', t.ho ro,-,o,,roh.T lo ron;. i .l..'f ,1 1 ,iiY,i' niiin - 

u|- v;ifi,il)lo;j ii.-. po;;;; il)lo i m;f:'. of ooiif;u; pon Ioi-i;i,hk:o wi t.hout, iii.ik 1 nc, 

t.lic iM..-,i.f i<;l.i,vo ,!.-.;uiiiipl 1011.-, roqiiLfoil hy ot.hiM' tochiiKiiK.'.s . In tlii^ ;um.'oiuI 
;ifo.t t.ho .li;;<;u.".;-.ion of tlio prohloiii;; in ^1 oiiion.-.fr.il. 1 nK, cuir.nt. i on ;ind the Miffi- 
,,,,H,J ,.,0, in,.-, Ih- l".'. inodol :;hoiiM h^Op t.lio."... involvod i_n ini|)r ov 1 m.', out' 
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